How do DeepL and ChatGPT process information structure and pragmatics?

An exploratory case study on topicalized infinitives in Spanish (and Portuguese)

Abstract

This case study focuses on a specific construction that exists in both Spanish and Portuguese, but not in English: topicalized infinitives (=TI), e.g., Sp. comer no come ‘as for eating s/he does not eat’. We present three pilot experiments: the first one is a translation task which consists of translating sentences with TI from Spanish to Portuguese and vice versa. DeepL failed in most cases due to contamination by English as a pivot language. The second task is a continuation task: ChatGPT-3.5 was asked to complete sentences that start with a TI. In most cases, natural and adequate continuations starting with pero ‘but’ were generated. Since this task is based on predicting the most likely continuation, this result is not surprising, as this is exactly how the model works. Contrarily, ChatGPT-3.5 demonstrated a clear inability to perform well on the third task, which consisted of drawing pragmatic inferences from exactly the same examples containing a TI that encodes an adversative implicature.